Automatic image colorization is a particularly challenging problem. Due to the high illness of the problem and multi-modal uncertainty, directly training a deep neural network usually leads to incorrect semantic colors and low color richness. Existing transformer-based methods can deliver better results but highly depend on hand-crafted dataset-level empirical distribution priors. In this work, we propose DDColor, a new end-to-end method with dual decoders, for image colorization. More specifically, we design a multi-scale image decoder and a transformer-based color decoder. The former manages to restore the spatial resolution of the image, while the latter establishes the correlation between semantic representations and color queries via cross-attention. The two decoders incorporate to learn semantic-aware color embedding by leveraging the multi-scale visual features. With the help of these two decoders, our method succeeds in producing semantically consistent and visually plausible colorization results without any additional priors. In addition, a simple but effective colorfulness loss is introduced to further improve the color richness of generated results. Our extensive experiments demonstrate that the proposed DDColor achieves significantly superior performance to existing state-of-the-art works both quantitatively and qualitatively. Codes will be made publicly available.
translated by 谷歌翻译
联合学习(FL)是一个有前途的分布式框架,用于协作人工智能模型培训,同时保护用户隐私。引起大量研究关注的引导组件是激励机制刺激佛罗里达用户协作的设计。大多数作品采用以经纪人为中心的方法来帮助中央运营商吸引参与者并进一步获得训练有素的模型。很少有作品认为参与者之间以参与者为中心的合作来追求其共同利益的FL模型,这会引起以经纪人FL的激励机制设计的显着差异。为了协调自私和异质参与者,我们提出了一个新颖的分析框架,以激励以参与者为中心的FL有效,有效的合作。具体而言,我们分别提出了两个新型游戏模型,用于贡献符合贡献的FL(COFL)和贡献感知的FL(CAFL),后者在其中实现了最低贡献阈值机制。我们进一步分析了COFL和CAFL游戏的NASH平衡的独特性和存在,并设计有效的算法以实现平衡溶液。广泛的绩效评估表明,COFL中存在自由骑行现象,通过采用CAFL模型具有优化的最低阈值,可以极大地缓解这种现象。
translated by 谷歌翻译
图形神经网络(GNN)在许多基于图的应用程序中取得了巨大成功。但是,巨大的尺寸和高稀疏度的图表阻碍了其在工业场景下的应用。尽管为大规模图提出了一些可扩展的GNN,但它们为每个节点采用固定的$ k $ hop邻域,因此在稀疏区域内采用大型繁殖深度时面临过度光滑的问题。为了解决上述问题,我们提出了一种新的GNN体系结构 - 图形注意多层感知器(GAMLP),该架构可以捕获不同图形知识范围之间的基本相关性。我们已经与天使平台部署了GAMLP,并进一步评估了现实世界数据集和大规模工业数据集的GAMLP。这14个图数据集的广泛实验表明,GAMLP在享有高可扩展性和效率的同时,达到了最先进的性能。具体来说,在我们的大规模腾讯视频数据集上的预测准确性方面,它的表现优于1.3 \%,同时达到了高达$ 50 \ times $ triending的速度。此外,它在开放图基准的最大同质和异质图(即OGBN-PAPERS100M和OGBN-MAG)的排行榜上排名第一。
translated by 谷歌翻译
K-Core Deconnosition是一个常用的指标来分析图形结构或研究节点在复杂图中的相对重要性。近年来,图表的规模迅速增长,特别是在工业环境中。例如,我们的工业伙伴以数十亿用户运行流行的社交应用程序,并且能够收集丰富的用户数据。因此,对大型图形的k核分解应用于学术界和行业的越来越多的关注。处理大图的简单但有效的方法是在分布式设置中训练它们,并且还提出了一些分布式k核分解算法。尽管他们有效性,我们在实验和理论上观察到这些算法消耗了太多资源,并在超大型图表上变得不稳定,特别是当给定的资源有限时。在本文中,我们处理那些超大型图形,并在分布式K核分解算法的顶部提出了分行和征服策略。我们在三个大图中评估我们的方法。实验结果表明,资源的消耗可以显着降低,大规模图的计算比现有方法更稳定。例如,分布式K-Core分解算法可以缩放到具有1360亿边缘的大图,而不会与我们的分行和征服技术丢失正确性。
translated by 谷歌翻译
Graph神经网络(GNN)最近在许多基于图的应用程序中都实现了最先进的性能。尽管具有很高的表现力,但他们通常需要在多个培训时期进行昂贵的递归邻里扩展,并面临可伸缩性问题。此外,它们中的大多数是不灵活的,因为它们仅限于固定跳跃社区,并且对不同节点的实际接受场需求不敏感。我们通过引入可扩展且灵活的图表多层感知器(GAMLP)来规避这些限制。随着非线性转化和特征传播的分离,GAMLP通过以预先计算的方式执行传播程序来显着提高可伸缩性和效率。有了三个原则的接受场注意力,GAMLP中的每个节点都具有灵活性和适应性,以利用接收场的不同尺寸的传播特征。我们对三个大型开放图基准(例如OGBN-PAPERS100M,OGBN产品和OGBN-MAG)进行了广泛的评估,这表明GAMLP不仅可以实现前面的性能,而且还提供了较高的可扩展性和效率。
translated by 谷歌翻译
机器学习系统通常假设训练和测试分布是相同的。为此,关键要求是开发可以概括到未经看不见的分布的模型。领域泛化(DG),即分销概括,近年来引起了越来越令人利益。域概括处理了一个具有挑战性的设置,其中给出了一个或几个不同但相关域,并且目标是学习可以概括到看不见的测试域的模型。多年来,域概括地区已经取得了巨大进展。本文提出了对该地区最近进步的首次审查。首先,我们提供了域泛化的正式定义,并讨论了几个相关领域。然后,我们彻底审查了与域泛化相关的理论,并仔细分析了泛化背后的理论。我们将最近的算法分为三个类:数据操作,表示学习和学习策略,并为每个类别详细介绍几种流行的算法。第三,我们介绍常用的数据集,应用程序和我们的开放源代码库进行公平评估。最后,我们总结了现有文学,并为未来提供了一些潜在的研究主题。
translated by 谷歌翻译
As one of the most important psychic stress reactions, micro-expressions (MEs), are spontaneous and transient facial expressions that can reveal the genuine emotions of human beings. Thus, recognizing MEs (MER) automatically is becoming increasingly crucial in the field of affective computing, and provides essential technical support in lie detection, psychological analysis and other areas. However, the lack of abundant ME data seriously restricts the development of cutting-edge data-driven MER models. Despite the recent efforts of several spontaneous ME datasets to alleviate this problem, it is still a tiny amount of work. To solve the problem of ME data hunger, we construct a dynamic spontaneous ME dataset with the largest current ME data scale, called DFME (Dynamic Facial Micro-expressions), which includes 7,526 well-labeled ME videos induced by 671 participants and annotated by more than 20 annotators throughout three years. Afterwards, we adopt four classical spatiotemporal feature learning models on DFME to perform MER experiments to objectively verify the validity of DFME dataset. In addition, we explore different solutions to the class imbalance and key-frame sequence sampling problems in dynamic MER respectively on DFME, so as to provide a valuable reference for future research. The comprehensive experimental results show that our DFME dataset can facilitate the research of automatic MER, and provide a new benchmark for MER. DFME will be published via https://mea-lab-421.github.io.
translated by 谷歌翻译
Interview has been regarded as one of the most crucial step for recruitment. To fully prepare for the interview with the recruiters, job seekers usually practice with mock interviews between each other. However, such a mock interview with peers is generally far away from the real interview experience: the mock interviewers are not guaranteed to be professional and are not likely to behave like a real interviewer. Due to the rapid growth of online recruitment in recent years, recruiters tend to have online interviews, which makes it possible to collect real interview data from real interviewers. In this paper, we propose a novel application named EZInterviewer, which aims to learn from the online interview data and provides mock interview services to the job seekers. The task is challenging in two ways: (1) the interview data are now available but still of low-resource; (2) to generate meaningful and relevant interview dialogs requires thorough understanding of both resumes and job descriptions. To address the low-resource challenge, EZInterviewer is trained on a very small set of interview dialogs. The key idea is to reduce the number of parameters that rely on interview dialogs by disentangling the knowledge selector and dialog generator so that most parameters can be trained with ungrounded dialogs as well as the resume data that are not low-resource. Evaluation results on a real-world job interview dialog dataset indicate that we achieve promising results to generate mock interviews. With the help of EZInterviewer, we hope to make mock interview practice become easier for job seekers.
translated by 谷歌翻译
Panoptic Part Segmentation (PPS) unifies panoptic segmentation and part segmentation into one task. Previous works utilize separated approaches to handle thing, stuff, and part predictions without shared computation and task association. We aim to unify these tasks at the architectural level, designing the first end-to-end unified framework named Panoptic-PartFormer. Moreover, we find the previous metric PartPQ biases to PQ. To handle both issues, we make the following contributions: Firstly, we design a meta-architecture that decouples part feature and things/stuff feature, respectively. We model things, stuff, and parts as object queries and directly learn to optimize all three forms of prediction as a unified mask prediction and classification problem. We term our model as Panoptic-PartFormer. Secondly, we propose a new metric Part-Whole Quality (PWQ) to better measure such task from both pixel-region and part-whole perspectives. It can also decouple the error for part segmentation and panoptic segmentation. Thirdly, inspired by Mask2Former, based on our meta-architecture, we propose Panoptic-PartFormer++ and design a new part-whole cross attention scheme to further boost part segmentation qualities. We design a new part-whole interaction method using masked cross attention. Finally, the extensive ablation studies and analysis demonstrate the effectiveness of both Panoptic-PartFormer and Panoptic-PartFormer++. Compared with previous Panoptic-PartFormer, our Panoptic-PartFormer++ achieves 2% PartPQ and 3% PWQ improvements on the Cityscapes PPS dataset and 5% PartPQ on the Pascal Context PPS dataset. On both datasets, Panoptic-PartFormer++ achieves new state-of-the-art results with a significant cost drop of 70% on GFlops and 50% on parameters. Our models can serve as a strong baseline and aid future research in PPS. Code will be available.
translated by 谷歌翻译
In contrast to the control-theoretic methods, the lack of stability guarantee remains a significant problem for model-free reinforcement learning (RL) methods. Jointly learning a policy and a Lyapunov function has recently become a promising approach to ensuring the whole system with a stability guarantee. However, the classical Lyapunov constraints researchers introduced cannot stabilize the system during the sampling-based optimization. Therefore, we propose the Adaptive Stability Certification (ASC), making the system reach sampling-based stability. Because the ASC condition can search for the optimal policy heuristically, we design the Adaptive Lyapunov-based Actor-Critic (ALAC) algorithm based on the ASC condition. Meanwhile, our algorithm avoids the optimization problem that a variety of constraints are coupled into the objective in current approaches. When evaluated on ten robotic tasks, our method achieves lower accumulated cost and fewer stability constraint violations than previous studies.
translated by 谷歌翻译